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Cyber  Intelligence  Tradecraft  Project 
Challenges  and  Best  Practices 
Cyber  Intelligence  Research  Consortium 
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Cyber  Intelligence  Tradecraft  Project 


Sponsor 

•  National  Intelligence  Manager  for  Cyber,  Office  of  the  Director  of  National 
Intelligence  (ODNI) 

Purpose 

•  Study  how  organizations  from  industry,  government,  and  academia 
perform  cyber  intelligence  (methodologies,  processes,  tools,  and  training) 

Definition  of  cyber  intelligence 

•  The  acquisition  and  analysis  of  information  to  identify,  track,  and  predict 
cyber  capabilities,  intentions,  and  activities  to  offer  courses  of  action  that 
enhance  decision  making 

Overall  finding 

•  The  most  effective  organizations  balanced  the  need  to  protect  their 
network  perimeters  with  the  need  to  look  beyond  them  for  strategic  insights 
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Q:  How  do  you  do  cyber  intelligence? 

“We  try  to  mirror  the 
traditional  intelligence 

cycle.” 

-  US  government  participant 


Stale 

processes 
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Traditional  Intelligence  Cycle 


Image  source:  ODNI  -  httD://www.dni.aov/index.phD/newsroom/reDorts-and-Dublications/193-reDorts-oublications-2013/835-u-s- 
national-intelliaence-an-overview-2013-SDonsored-bv-the-intelliaence-communitv-information-sharing-executive 
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Reporting  timelines 


Urgent 

Normal 

Strategic 

Gov’t  Agency  1 

2-4  Hours 

1  Day 

1  Month 

Gov’t  Agency  2 

1  Day 

2  Weeks 

3  Months 

Gov’t  Agency  3 

1  Day 

3  Months 

6-18  Months 

Gov’t  Agency  4 

2  Hours 

8  Hours 

5  Days 
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Success  using  nonlinear,  interactive 
conceptual  frameworks 


Analytical  Acumen 


Environmental 

Context 


Data 

Gathering 


Reporting  & 
Feedback 


Analytical 

Acumen 


Macroanalysis 


Microanalysis 


•  Facilitates  timely/actionable/accurate  intelligence 

Environmental  Context 

•  Provides  scope  for  the  analytical  effort 

Data  Gathering 

•  Acquires  and  aligns  data  for  analysis 

Microanalysis 

•  Assesses  functional  implications 

Macroanalysis 

•  Assesses  strategic  implications 

Reporting  and  Feedback 

•  Offers  courses  of  action  to  enhance 
decision  making 


•  /A°Or* 

•  .^ent 
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Q:  How  do  you  rank  threats,  from  high  to  low? 

“We  consider 
everything  a  high 
priority  threat.” 

-  US  government  participant 


Stale  Threat 

processes  prioritization 
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Capability 


Implementing... 

Threat  =  Potential  +  Impact  +  Exposure 
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Operations 


Cyber  Footprint 
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Infrastructure 

Operational  structures  needed 
for  success — hardware, 
software,  or  command  and 
control 

Technology 

Whether  used  or  manipulated 

Coding 

Nuances  and  personal 
preferences 

Maturity 

According  to  the  planning 
process  and  pre/ post-threat 
activities 


—  Money 

For  personnel,  tools,  training, 
or  access 

—  People 

Number  and  type  of  people 
involved — collaborators, 
teachers,  mentors,  or 
sponsors 

—  Tools 

Open  source  and/or  custom, 
and  why 

Training 

Type  and  quality 


—  Intrinsic 

Personal  rewards  to  act  on  the 
threat — bragging  rights, 
knowledge,  justify  skills, 
satisfy  boredom,  patriotism,  or 
hacktivist  allegiance 

Extrinsic 

External  rewards  to  act  on  the 
threat — fame,  money — or  to  avoid 
punishment 


—  Personally  Identifiable 
Information  (Pll) 

Payment  card  data,  social 
security  numbers,  or 
biometrics 

—  Organizational  Data 

Research  and  development 
information,  business 
processes,  or  industrial 
control  systems 


—  Targets 

General  or  specific — mass 
phishing  data  or  exploiting  a 
specific  vulnerability 

L  Timing 

Minutes,  days,  or  years  to  act 
on  the  cyber  threat 
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Incident  Response 

Costs  to  perform  an 
investigation,  remediation, 
and  forensics 

Downtime 

Business  costs  of  a 
network-reliant  service  being 
unavailable — missed  financial 
transactions  or  loss  of 
potential  product/services 
revenue 

Mitigation  and/or 
Prevention 

Costs  of  additional 
hardware/ software  to  stop 
current  and  future  threats 


Supply  Chain 

Costs  associated  with  the 
inability  to  meet  demand, 
delay  to  operations,  and 
supplementing  or  replacing 
suppliers 

Logistics 

Cost  of  continuing  business 
operations  during  and  after  an 
attack — rerouting 
communications,  securing 
intellectual  property,  or 
upgrading  processes 

Future  Earnings 

How  the  threat  affects  R&D, 
product  releases, 
acquisitions,  or  competitive 
advantage 


Strategic  Planning 

How  the  threat  affects  the 
strategic  vision — annual 
reports,  operational  policies, 
or  mergers 

Stakeholders 

Threat  impact  on  shareholders, 
board  of  directors,  or 
employees 

Culture 

How  the  threat  affects 
legal/regulatory  requirements, 
network  access,  or 
work-from-home  policies 


Market/Industry 

Threat  impact  on  target's 
competitors  and  industry,  both 
domestic  and  foreign 

Geopolitical 

How  the  threat  affects  political 
relationships  and 
local/ national/global 
economies 

Partnerships 

Threat  impact  on  target’s  third 
party  providers,  information 
sharing  agreements,  or  other 
business  relationships 

Brand  Reputation 

How  the  threat  affects  the 
target’s  brand  and  its 
implications  on  public  opinion 
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OM1 


Relevance 


—  Internet  Presence 

Susceptible  witting  and 
unwitting  information 
target-related  individuals  put 
online  and  their  popularity  on 
blogs/social  media 

—  Extracurricular  Activities 

Vulnerabilities  from  these 
individuals  roles  with 
non-target 

entities — non-profits, 
activist  groups,  or 
local/national  politics 

L  Motive 

The  reasons  for  why  such 
individuals  are  susceptible  to 
the  cyber  threat — ignorance, 
financial  trouble,  disgruntlement, 
or  boredom 


Access 


—  Physical 

Vulnerabilities  from 
target-related  individuals 
ability  to  access  the  target’s 
tangible  aspects — office 
space,  transportation,  or 
equipment 

—  Network 

Susceptible  administrative 
privileges  or  sensitive  data 
access  provided  to  such 
individuals 

—  Position 

How  threat  actors  exploit 
the  different  roles  these 
individuals  play  for  the 
target — network  administrator, 
senior  leader,  or  rank-and-file 
employee 

Abnormal  Activities 

Deviations  from  normal 
physical,  network,  or 
position-based  activities  of 
key  target-related  individuals 
can  signify  potential  vulnera¬ 
bilities 


I 

Infrastructure 


—  Hardware 

Risks  emanating  from  where 
network  appliances, 
workstations,  and  third  party 
equipment  connect  to  the 
target's  network 

-  Software 

Risks  associated  with  the 
target  relying  on  particular 
software  for  day-to-day 
operations,  providing  access 
to  high-risk  software,  and 
detecting  software  vulnerability 
exploitation 

Supply  Chain 

How  the  cyber  threat  affects 
the  target’s  acquisition, 
implementation,  maintenance, 
and  discontinuation  of 
hardware  and  software 


Internet  Presence 


—  Website 

How  the  threat  actor  can 
leverage  the  target’s 
website — compromise 
content,  collect  data,  or 
deny  access 

—  Social  Media 

Risks  associated  with  the 
target’s  use  of  it  for  organiza¬ 
tional  activities — 
marketing,  customer  service, 
or  product  placement 

—  Additional  Services 

Risks  emanating  from  the 
target’s  use  of  FTP,  Telnet, 
VPN,  webmail,  remote 
desktop,  and  other  web-based 
services 
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Q:  Where  do  your  decision  makers  generally 
get  their  cyber  intelligence? 

“CNN.” 


-  Financial  sector  participant 


Stale  Threat  Communicating  to 

processes  prioritization  decision  makers 
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Validity  of  cyber  intelligence  partnerships 


Cyber  Intelligence 
Partnership 

Business  Intelligence 
and 

Cyber  Intelligence  Program 


INTRODUCTION . 1 

Business  Intelligence  Mission  Statement . 1 

Cyber  Intelligence  Program  Mission  Statement . 1 

PURPOSE . 2 

SCOPE . 2 

REVIEW  AND  EVALUATION . 3 

SHARING  OF  SENSITIVE  INFORMATION . 4 

CYBER  INTELLIGENCE  PROCESS . 5 

EXISTING  INTELLIGENCE  GAP  REQUESTS . 10 

APPENDIX  1:  TEMPLATES . 13 

APPENDIX  2:  DETAILED  INTERACTION  PROCESSES . 16 

APPENDIX  3:  INTELLIGENCE  GAP  REQUEST  WORKFLOW . 17 

APPENDIX  4:  MEETING  AGENDAS . 18 

APPENDIX  5:  GLOSSARY  OF  TERMS  AND  ACRONYMS . 20 
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Q:  How  do  you  demonstrate  return  on 
investment? 

“We  don’t.” 


-  Energy  sector  participant 


Stale  Threat  Communicating  to  Return  on 

processes  prioritization  decision  makers  Investment 
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Compare  and  contrast  for  ROI 


Anonymous  Message  to  The  University  Of  Pittsburgh 

Image  source:  https://www.  youtube. com/watch?v=X1  Tgbdl mi  U 
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Q:  Can  you  describe  your  data  collection 
process? 

“It’s  an  absolute 
mess...” 


-  Energy  Sector  Participant 


Stale  Threat  Communicating  to  Return  on  Collection 

processes  prioritization  decision  makers  Investment  management 
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Levels  of  collection  management 


Basic 

Established 

Advanced 

Requirements 

•  Establish  collection 
mechanisms 

•  Identify  stakeholders 

•  Add  rigor:  Not  all  requests 
are  created  equal 

•  Classify  requirements 

•  Track  requirements 

•  Incorporate  needs  of  all 
stakeholders 

•  Continually  validate 
requirements 

Operations 

•  Know  your  data  sources 

•  Assess  and  manage 

•  Validate  and  evaluate  third 

•  Know  your  information  gaps 

sources 

party  information 

•  Align  data  with  requirements 

•  Validate  data  quality  and 
reliability 

•  Ensure  redundancies  exist 
for  data  coverage 

•  Look  beyond  network  data 

•  Let  intelligence  drive  data 
collection 

•  Leverage  tipping/queuing 

Analysis  & 

•  Collect  data,  fuse  sources 

•  Corroborate  information 

•  Anticipate  requirements 

•  Add  context  and  calculated 

with  multiple  sources 

•  Automate  analysis  of 

Reporting 

judgments/predictions 

•  Ensure  priority 
requirements  are  being  met 
with  the  available  data 

sources 

known  threats 
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Establishing  an  evaluation  cycle 


Intelligence  Report 


Please  click  here  to  provide 
us  with  feedback 


Intelligence  providers  are 
evaluated  based  on  the 
feedback  from  the  reports. 


Intelligence  received  from 
various  sources  is  compiled 
and  distributed  to  stakeholders. 


Each  report  has  a  feedback 
link  where  consumers 
evaluate  timeliness, 
usefulness,  and  actionability. 


Key  Performance 
Indicators 

FEEDBACK 

-  H  m  r\ 

™ m 

O 

o  ■■■ 

The  feedback  is 

o  ■■ 

aggregated  and  indirectly 

scores  the  quality  of  the 

intelligence  source. 
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Evaluating  Intelligence 


Challenge 

•  Cyber  intelligence  is  a  phrase  often  used,  but  interpreted  in  many 
different  ways,  leading  to  a  diverse  output  of  threat  analysis 
categorized  as  cyber  intelligence 

•  Such  output  is  difficult  to  evaluate  and  compare,  stifling  an 
organization’s  ability  to  establish  guidelines  and  goals 

Solution 

•  An  evaluation  template  based  on  standards  observed  during  our 
research  and  set  forth  in  U.S.  Intelligence  Community  Directive  Number 
203 

•  http://www.dni.gov/files/documents/ICD/ICD%20203%20Analvtic%20Sta 

ndards%20pdf-unclassified.pdf 
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Template  -  Evaluating  Intelligence 


Assess  the  quality  and  thoroughness  of  an  intelligence  analyst’s 
work  using  a  grading  system  based  on  points  accumulated  for 
criteria  the  analyst  satisfies  in  an  intelligence  product 

Grading  system 

A:  17-16,  B:  15-14,  C:  13-12,  D:  11-10,  F:  9  and  below 
Criteria 

•  Objective 

•  Independent  of  political  considerations 

•  Timely 

•  Based  on  all  available  sources 

•  Exhibiting  proper  standards  of  analytic  tradecraft 
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Cyber  Intelligence  Research  Consortium 


Purpose 

•  Research  and  develop  technical  solutions  and  analytical  practices  to  help 
people  make  better  judgments  and  quicker  decisions  with  cyber  intelligence 

Membership 

•  Decision  makers  and  practitioners  from  academia,  Department  of  Defense, 
defense  contracting,  energy,  financial  services,  and  the  U.S.  Intelligence 
Community 

Offerings 

•  Cyber  threat  baseline:  Threat  environment  research  to  identify  best  practices 

•  Tradecraft  labs:  Workshops  to  advance  analytical  &  technological  capabilities 

•  Implementation  frameworks:  How-to  guides  for  key  intelligence  practices 

•  Crisis  simulation:  Capture-the-flag  exercise  to  apply  techniques  &  technologies 

•  Intelligence  insights:  Continuous  communication  on  relevant  topics 
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Questions? 

Jay  McAllister 
412.268.9193 
iimcallister@sei.cmu.edu 

@sei_etc 

Output  from  Cyber  Intelligence  Tradecraft  Project 

•  http://www.sei.cmu.edu/about/organization/etc/citp.cfm 

Information  on  the  Cyber  Intelligence  Research  Consortium 

•  http://www.sei.cmu.edu/about/organization/etc/overview.cfm 
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